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Abstract — We consider a fundamental game theoretic problem 
concerning selfish users contributing packets to an M/M/1 queue. 
In this game, each user controls its own input rate so as 
to optimize a desired tradeoff between throughput and delay. 
We first show that the original game has an inefficient Nash 
Equilibrium (NE), with a Price of Anarchy (PoA) that scales 
linearly or worse in the number of users. In order to improve 
the outcome efficiency, we propose an easily implementable 
mechanism design whereby the server randomly drops packets 
with a probability that is a function of the total arrival rate. We 
show that this results in a modified M/M/1 queueing game that is 
an ordinal potential game with at least one NE. In particular, for a 
linear packet dropping function, which is similar to the Random 
Early Detection (RED) algorithm used in Internet Congestion 
Control, we prove that there is a unique NE. We also show 
that the simple best response dynamic converges to this unique 
equilibrium. Finally, for this scheme, we prove that the social 
welfare (expressed either as the summation of utilities of all 
players, or as the summation of the logarithm of utilities of all 
players) at the equilibrium point can be arbitrarily close to the 
social welfare at the global optimal point, i.e. the PoA can be 
made arbitrarily close to 1. We also study the impact of arrival 
rate estimation error on the PoA through simulations. 

I. Introduction 

In the past twenty years, the usage of the Internet has 
I transitioned from being primarily academic/research-oriented 
' to one that is primarily commercial in nature. In the current 
Internet environment, each commercial entity is inherently 
■ interested only in its own profit. Developing network mecha- 
nisms that are designed to handle selfish behavior has therefore 
1 gained increasing attention in recent years. The game theoretic 
approach, which was originally designed to model and guide 
decisions in economic markets, provides a valuable set of tools 
for dealing with selfish behavior L2J-L7J- 

In this work, we consider the network congestion problem at 
a single intermediate store-and-forwarding spot in the network. 
Several users send their packets to a single server with Poisson 
arrival rate. The server processes the packets on a first come 
first serve (FCFS) basis with an exponentially distributed 
service time. This is an M/M/1 queueing model [8J. There 
exists a trade-off in this M/M/1 queueing model between 
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throughput (representing the benefit from service), and delay 
(representing the waiting cost in the queue). In the gateway 
congestion control context |9|, a measure that is widely used 
to describe this trade-off is called "Power", which is defined 
as the weighted ratio of the throughput to the delay. When the 
users are selfish, we can formulate a basic M/M/1 queueing 
game. In this game, we assume that the users are selfish, and 
each control their own input arrival rate to the server Each 
user's utility is modeled to be the power ratio for that user's 
packets. 

This classic M/M/1 queueing game has been formulated and 
studied in |,10J - IJ4j . The results from these prior works and 
our own results in this work are in agreement that the basic 
M/M/1 queuing game has an inefficient Nash Equilibrium. We 
are therefore motivated to design an incentive mechanism to 
force the users to operate at an equilibrium that is globally 
efficient. In particular, we focus on the design of a packet 
dropping scheme implemented at the server for this purpose. 
Our objective is that the dropping scheme should be as simple 
as possible, and it should minimize the Price of Anarchy (PoA, 
the ratio of the social optimum welfare to the welfare of the 
worst Nash equilibrium) to be as close to 1 as possible. 

A key contribution of this work is the formulation of a 
modified M/M/1 queuing game with a randomized packet 
dropping policy at the server We consider a simple and low 
overhead policy in our formulation, wherein the server need 
only monitor the sum of the rates of all users in the system. We 
show that this modified game with a packet dropping scheme 
is an ordinal potential game [15], which implies the existence 
of at least one pure Nash Equilibrium. 

We show first that utilizing a step-function for packet 
dropping whereby the server drops all the packets when the 
sum-rate is greater than a threshold (and none when the sum- 
rate is below the threshold), results in infinite number of 
undesired Nash Equilibria which harms the PoA. 

This raises the question whether a more sophisticated ap- 
proach can do better. We show that indeed this is possible. 
In particular, we develop an incentive mechanism with a 
linear packet dropping that can improve the system efficiency 
to be arbitrarily close to the global optimal point (i.e., a 
PoA arbitrarily close to 1). This mechanism is similar to the 
Random Early Detection (RED) used for congestion avoidance 
on the Internet |fT6l . We prove the uniqueness of NE of the 
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game with this mechanism. We also show that best response 
dynamics will converge to the unique NE. 

Our paper is organized as follows. Section summarizes 
the related work. We present the model of an M/M/1 queue 
game in Section |III] The social welfare and Price of Anarchy 
are described in section |IV] to investigate the efficiency of 
the NE. Then, in section [V] we propose to design an incentive 
packet dropping scheme implemented at the server to improve 
the efficiency. Section [VT] proves that the game defined with 
packet dropping policy is an ordinal potential game by giving 
the potential function. Section IVIII shows the best response 
function. In section IVlIIl we show the behavior when utilizing 
a simple step-function for packet dropping. In section |IX] we 
propose the RED-like linear packet dropping incentive scheme. 
We show that with this scheme, it is possible to make the 
Price of Anarchy arbitrarily close to the optimal point. The 
uniqueness of NE of such a game is proved in section |X] 
In section IXj we show that the best response dynamics will 
converge to the unique Nash Equilibrium. In section IXIII we 
undertake simulations to see how the process of statistically 
estimating the input arrival rates in a real system would impact 
the PoA. We conclude the work in section IXIIII 

II. Related Work 

Throughput-delay tradeoffs in M/M/1 queues with selfish 
users have been previously studied in 1 10|-[ 14|. A utility func- 
tion for each user is defined as the corresponding application's 
power and each user is treated as a player in such a game 
and adjusts its arrival rate to handle the trade-off between 
throughput and delay. Every user is assumed to be selfish and 
only wants to maximize its own utility function in a distributed 
manner. 

Bharath-Kumar and Jaffe fTOl wrote one of the earliest 
papers on the formulation of throughput-delay tradeoffs in 
M/M/1 queues with selfish users. The paper discusses the 
properties of power as a network performance objective func- 
tion. A class of greedy algorithms where each user updates its 
sending rate synchronously to the best response of all other 
users' rates to maximize the power is proposed. Convergence 
of the best response to an equilibrium point is shown in this 
paper 

Douligeris and Mazumdar lilli extended Bharath-Kumar 
and Jaffe's work to the case with different weighting fac- 
tors defined in the power function for different users and 
provided analytical results describing the Nash Equilibrium. 
They showed that the equilibrium point that the greedy best 
response dynamic algorithm converged to was a unique Nash 
Equilibrium. 

The work by Zhang and Douligeris |12| proved the con- 
vergence of the best response dynamics for this basic M/M/1 
queueing game under the multiple users case. Thus all these 
prior works ( lfT0ll - lfT2l ) studied only variants of the basic 
game. Their work, along with ours, shows that this basic 
game results in an inefficient outcome. Our work is the first to 
develop a mechanism design for this problem that addresses 



this shortcoming by showing how to achieve near-optimal 
performance using a packet-dropping policy. 

Dutta et al. ||13 .| studied a related problem involving a server 
that employs an oblivious active queue management scheme, 
i.e. drops packets depending on the total queue occupancy 
with the same probability regardless of which flow they 
come from. They also consider an M/M/1 setting with users 
offering Poisson traffic to a server with exponential service 
time. The users' actions are the input rates and the utilities 
the goodput/output rates. The existence and the quality of 
symmetric Nash equilibria are studied for different packet 
dropping policies. Although our work also explores oblivious 
packet dropping schemes, it is different from and somewhat 
more challenging to analyze than [13], because our utility 
function reflects the tradeoff between goodput and delay. 

In another, more recent work, lfT4l . Su and van der Schaar 
have discussed linearly coupled communication games in 
which users' utilities are linearly impacted by their competi- 
tors' actions. An M/M/1 FCFS queuing game with the power 
as the utility function is one illustrative example of linearly 
coupled communication games. They also quantify the Price 
of Anarchy in this case, and investigate an alternative solu- 
tion concept called Conjectural Equilibrium, which requires 
users to maintain and operate upon additional beliefs about 
competitors. 

There have been also several other papers related to queue- 
ing games, albeit with different formulations. Haviv and 
Roughgarden |17| considered a system with multiple servers 
with heterogeneous service rates. Arrivals from customers are 
routed to one of the servers, and the routing decisions are 
analyzed based on NE or social optimization schemes. PoA is 
shown to be upper bounded by the number of servers for the 
social optimum. Wu and Starobinski |18| analyzed the PoA 
of N parallel links where the delays of links are characterized 
using unbounded delay functions such as M/M/1 or M/G/1 
queueing functions. Economides and Silvester |[T9l studied 
a multiserver two-class queueing game and developed the 
routing policy. 

For more general surveys on game theoretic formulations 
of networking problems, we refer the reader to HJ, ll20l . 

III. Problem Formulation 

We consider a M/M/1 FCFS queue game as shown in Fig. 
[T] There are m users with independent Poisson arrivals and 
the arrival rates are Ai, A2, . . . , \m- There is a single server 
and the service time is exponentially distributed with mean i. 

We consider each user as a player for this game and the 
users are selfish. Each player wants to maximize its own utility 
function by adjusting its rate sending to the queue. 

Note that there is a tradeoff between the throughput and 
delay for each user, i.e., given the rates of all other users, 
if the input rate increases, the delay increases too. In this 
paper, we consider the measurement of this tradeoff between 
the throughput and delay of the each user, and it is known as 
the "power", which is widely used in the gateway congestion 
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Fig. 1. An M/M/1 queue 

control context f9l. We consider the power as the utility 
function of each user to measure its throughput-delay tradeoff. 
For a given user i, the power is defined as: 

Throughput"' 



Power 



(1) 



Delay 

where Ui is a parameter chosen based on the relative emphasis 
placed on throughput versus delay, > 1 when throughput is 
more important, while < ai < 1 when we want to emphasis 
delay more, and ai — 1 when the throughput and delay are 
emphasized equally. 

For M/M/1 queue, the throughput for user i \s Ti — 
where is the effective rate served by the server. The delay 
for user i is calculated as: Di = i — . So the power for 



user i can be expressed as: 



i = l 



i=l 



A-)- 



(2) 



In this M/M/1 game, each player is selfish and wants to 
adjust its arrival rate Ai to maximize its own utility function. 
Throughout the paper, we assume that the queue is stable and 

m 

thus < Af < A*- 

i=l 

When there is no dropping policy implemented at the server, 
\l = \i, and the optimization problem for each player i is: 



max 



s.t. 



E 

i=l 



(3) 



2^ Ai < /i 
Ai > Vi = l,2,...,m 

IV. Social Welfare and Price of Anarchy 

In ifTTl the above M/M/1 queue game is studied and a unique 
pure NE is proved to be: 



Af^ = 



(4) 



fe=i 



When tti = a,Vz, this unique NE is expressed as = 
am+i ' Now suppose all users cooperate to achieve the 
maximal system utility. We consider two ways to define the 
social optimal function: the sum of the utilities of all the 
users and the sum of the logarithm of the utilities of all the 
users. Defining the social optimal function as the sum of the 
utilities of all the users is a common way for evaluating the 
system efficiency and we present the analysis results under this 
definition first. However, the fairness among the users should 



also be considered and it is not revealed under this definition; 
so we also consider a log-sum-utility social welfare function 
which provides for utility fairness. 

We can measure the efficiency of the system using two well 
known measures called the Price of Anarchy (PoA) and Price 
of Stability (PoS), that respectively compare the performance 
of selfish users in the worst and best case Nash Equilibrium 
with the global optimum achievable with non-selfish users. 
The definition of PoA and PoS of a game G is: 



PoA(G) = max 

ae£{e 

PoS{G) = min 



ae£{G) U{a) 



(5) 



(6) 



ae£(G) U{a) 

where £ is the set of all the Nash Equilibriums in game G. 

A. Sum-utility 

The optimization problem is defined as: 

max EAf(M-EA») (7) 

s.t. < E Aj < /i 

A,; > Vi = l,2,...,m 

Here we consider two cases: 
1) a > 1 

First calculate U'^^^. 



C/^-- = m^ax)^A?(M-)^A,) 

< inax(^ A,)"(^ ~ E 
— max A"(/i — A) 

A 



(8) 



We can get X* ^ -li^. 

Equality holds in equation [8] when Ai = A* for some i, and 
Ai = 0,Vj 7^ i. Hence this is also the solution for P^^^ . 



U 



OPT _ a M 



Then we calculate JJ when players are selfish: 



U 



{am + 

Note that there is only one NE in the game, so PoA and 
PoS are the same and they are derived as below: 

f/C^^ (am + 

In this case we find that the PoA and PoS are proportional 
to m". 

2) a < 1 

The calculation is similar as above, and details are omitted. 

C/0^^ = m^ax^A?(,.-5:A.) 

< mma.xfi^)°'(^l - V A,) 



maxmi-"A"(/i - A). 

A 
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We also get A* = 



jjOPT^ 

PoA and PoS are: 

PoA{G) = PoS{G) = 



(a + l)"+i ■ 

U^P^ _ (am + l)°+i 
-JfNE- 



(10) 



to" (a 

In this case we find that the PoA and PoS are proportional 

to TO. 

Thus in both cases, we find that the PoA and PoS degrade 
linearly or worse with the number of users. 

B. Sum-log-utility 

Now let's consider the sum of the logarithm of the utilities 
of all the users. The reason we consider the logarithm function 
in the social welfare is because when all users cooperate to 
achieve the optimum, fairness among the users should also be 
considered, and a logarithmic function would ensure this. The 
social welfare optimization problem is: 



max 



s.t. 



i=l 



log 



(11) 



i=l 

> Vi 



1,2,. ..,m 



Note that for each player, maximizing the logarithm of its 
utility function is equivalent to maximizing the utility function 
itself. Therefore the NE remains the same as before. 

m 

Denote A = J2 ^i- We have the following theorem for 

i=l 

finding out the social optimum: 

Theorem 1: The solution for the social welfare optimiza- 
tion problem is: A* = ) . 

Proof: see Appendix lAl ■ 

Note that is shown in (|4|i and by substituting it into 
(O, we get the power for user i as: 



NE 



[am + 



In general, the log-utility terms can be negative. To ensure 
that both the numerator and denominator terms in the PoA 
and PoS are non-negative in this case, we use a monotonic 
exponential mapping. Note that there is only one NE in the 
game, so PoA and PoS are the same, and they are derived as 
below: 



PoA{G) = PoS{G) 



rOPT ( 

\^m°(Q+l)°-< 

(Qm+1)° + 

[am + 



(12) 



> 1. 



^TO"(a + l)"+\ 

From (fTZt we can see that PoA increases monotonically as 
TO increases and goes to infinity as m goes to infinity. So we 



want to implement an incentive mechanism to improve the 
PoA. 

V. An Incentive Packet Dropping Scheme 



Note that A, 



NE 



> 



A*. This inspires us 



Qm+l ^ m(a+l) 

to find an incentive packet dropping mechanism implemented 
at the server and we wish this packet dropping mechanism to 
be as simple as possible. So we consider the case where the 
server need only monitor the sum of the rates of all users in 
the system and implement the packet dropping policy based 
only on this information. Then the packet dropping function 
could be expressed as PdiYl ^i)- So the optimization problem 
for each user i with a dropping policy in the system is: 



s.t. 



[/,(A„A_,) - (A,(l-Pd(;^A,)))"' 

(m-5](a,(i-p,(^a,)))) 

^A,(1-P,(^A.))<M 
Ai>0 Vi = l,2,...,TO 



(13) 



To facilitate the derivation, denote P{J2 ^0 — l^^d(X) ^i) 
and thus P(-) is the probability of keeping packets in the 
system. Then the optimization problem for each player i is: 

max C/,(A„A_0 = (A,P(5]A,)r 

s.t. ^A,P(^A,)<M (15) 

71 (16) 



Ai > 



1,2,. 



We denote the above game as Gp — {N, {Ai}, {Ui}). 

VI. Potential Game 

In this section, we prove that when the dropping function is 
a function which only depends on the sum of total incoming 
rates, the game is a potential game and thus there exists at 
least one pure NE. 

Definition 1: a game G = {N,{Ai},{Ui}) is called an 
ordinal potential game if there exists a global function : 
A — > II such that for every player i ^ N, for every 
a_i € A-i and for every a,^, a" € Ai, 

sgn{Ui{a'^,a^i)~Ui{a'-,a^i)) = s.gn((/)(a-, a_i)-(/)(a-', a^^)) 

(17) 

where sgn{x) is the sign function that takes on the value -1 
when a: < 0, when a; = 0, and 1 when a; > 0. 

Also, the following Theorem |2] holds for the existence of 
NE in a potential game: 

Theorem 2: [Monderer-Shapley, 1996 |15|] Every potential 
game with finite-players, continuous utilities, and compact 
strategy sets possesses at least one pure-strategy equilibrium. 

Now we will prove that the M/M/1 queueing game with a 
packet dropping function Pd(X] ^0 ^ potential game. 

Theorem 3: Gp is a ordinal potential game with potential 
function 
(?!)(Ai, A2, . . . , A„,) 



n(A.P(EA.))- 

! = 1 
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Proof: 

</.(AU-0-'/'(Ar,A_.) 

' m 



n(A.p(EAor [(M-m+A_. 



(E^^ + ^^))(W'. + A-or 



if, p{x'i + A_o(E + An)(Arp(Ar + A-o)"i 



= \ X\{>^^pC£^^)r I (t/,(AU-.)-c/.(Ar,A-.)) 

Note that Gp has a finite number of players and continuous 
utiHties. However its strategy sets are not compact in ( fTSl l so 
we could not directly apply Theorem |2] to claim there exists at 
least one NE in Gp. But we modify the Gp to be the equivalent 
game as follows: 



max 



s.t. 



C/,(A„A_,) = (A,P(E A,)))"- 

(m-E(a,p(Ea.)))) 

Y^KPiY^X.) < fji 
A,; > Vi = l,2,...,m 



(18) 



since any solution to the maximization problem in Gp will 
not satisfy X]Ai^'(X]Ai) = A*- Now strategy sets of Gp are 
compact and thus there exists at least one NE in Gp. 

Note that in the following when we describe PoA and PoS 
for the packet dropping game Gp, we respectively compare the 
worst and best NE obtained for this game with respect to the 
social welfare (global optimum) that can be obtained through 
cooperation without packet dropping. 

VII. Best Response Function 

From now, for tractability, we consider the case = a, Vi 
for our proposed incentive packet dropping scheme. 
Vi, let 

9;7,(A„A'_ 



dX, 



0. 



If P(X] Ai) is differentiable with respect to A;, we will have 

aPfi - (a + 1)PP'A,A_, - [a + 1)PP'X^ 



aP^X-^ - (a + l)P^Ai + aP'X.fi = 



(19) 



where P' is the derivative of P{J2 Ai) with respect to A^. 

The above defines an implicit best response function 
d{Xi, X-i) — which shows the relationship between Ai and 

A-,. 

VIII. Step Dropping Function 

An intuitive dropping policy that first comes to mind is a 
step function as shown in Fig. |2] 
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Fig. 2. Step dropping function Pd(Yl ^i) 



The expression of PdiYl Aj) is: 



E A^ < A* 
E A, > A* 



(20) 



We have the following result for the corresponding packet 
dropping game. 

Theorem 4: A' is a N.E. if and only if ^ A^ = A*. 

Proof: see Appendix IbI ■ 
Based on Theorem |4] the NEs of the game with the step 
dropping function are not unique. PoS = 1 since there exists a 
NE with Ai = A*, Vi. However, in the sum-utility case, PoA = 

when a < 1. Moreover, 



m 



" ^ when a > 1, and PoA 



PoA is infinite in the sum-log-utility case since there exists a 
NE which has one user i with A^ = 0. Hence this is not 
a desirable result for improving the efficiency. We therefore 
next consider a slightly more sophisticated dropping function 
that has a linear profile. 

IX. Linear Dropping Function 



'1 '2 paT^^) 'l ''2 nl'i,) 

Fig. 3. Illustration of PdiT. and P{J2 Ai) 

We consider the game with the following linear function of 
P(E Ai) (and thus the packet dropping function Pd(E Ai) — 
1 — P(EAi) is also a linear function.) Fig. |3] illustrates 
Pd(EA.) and P(EA.). 



PiJ2x^)-{ A(EA. 





D 



< E A^ < 

ri<EA,:<r-2 (21) 
E A^ > ^2 
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where 




(22) 



= 0. 



For linear dropping scheme, ( fT9] l becomes: 

aPfi -{a + l)PAXiX-i - {a + l)PAXf 
- aP'^X-i -{a + l)P^Xi + aAXifi = 

The above also defines an implicit function ^{Xi, A_ 

Denote Xf = P{J2 A»)Aj and Xt, = P(E Aj)A_j. First we 
want find out if we could design a dropping policy in this 
linear form such that the system could have a NE that is the 
same as the social optimum. If not, we will then explore how 
much efficiency it could achieve. 

Theorem 5: There does not exist a linear packet dropping 
policy such that PoA = 1. 
Proof: 

Assume the above Theorem does not hold, when P ^ 0, 
substituting A; and A_i with A^/P and X'Li/P we have. 



e\2 



aP^, - i(a + l)AXtXU -^{a + l)A{X1) 
- aPX% - (a + l)PX1 + ^aAXl^i = 
P(aM-aAl,-(« + l)An 
^h{a + 1)AX'IXU + {a + l)A{Xlf ~ aAX1^i] 



P 



Since A* 



impHes = (a + 1)A*, we have 

l)A*-aAl, -(a + l)A^] 
-AX1[{a + l){X% + X1)-{a 
X% - XI) + X%] 
-AXl(,a + l){XU + Xt-X*) 



1)A* 



(23) 



A* 



P[{a- 
1 

^ P' 
^ P[{a + l){X* 
_ 1 
" P' 

Note that PA^ + PA_, = A* impHes that Af + A^ ^ 
So the right-hand side of the equality is 0. While the left-hand 
side of the equality is PAlj. Since Al^ ^ 0, so PAlj ^ 0. 
Thus the left-hand side of the equality is not and this leads 
to a contradiction. Therefore Theorem |5] holds. ■ 

Theorem |5] shows that we could not design a linear packet 
dropping policy with PoA — 1. The following theorem shows 
that we could design an incentive packet dropping policy such 
that PoA could be arbitrarily close to 1. 

Theorem 6: Given any e, there exists a linear packet drop- 
ping policy such that 1 < PoA < 1 + e. 
Proof: 

Note ( l23T l in the proof of Theorem |5] implies: 

P\a + l){X*-XU-Xl) + XU] = AA^(a + l)(Ai, + A,^-A*) 

(24) 

The right-hand side of (l24l i is greater than only when 
Xl, + XI < X* (note that A < 0). Then given A and Af , Ai, 



such that Xti + Xf < A*, we will have a solution for P^ and 
thus we could get the value of D. 

This means that we can design a packet dropping scheme 
such that it has a NE that satisfies pXi + pX^i — > X* from 
the left side (left approximation). If we can further prove that 
the NE is unique in this game (see Theorem [8]i, then give any 
e > 0, we could find a linear packet dropping policy at the 
server such that 1 < PoA < 1 + e. 

■ 

We propose Algorithm [T] to show how to design the pa- 
rameters ri and r2 in our proposed incentive packet dropping 
policy to achieve a desired PoA such that 1 < PoA < 1 + e 
given any e. We denote Ae = ^ Af . Line [T] ensures that 
the sum of the rates will be less than /i before the server 
starts to drop packets, p is the value of P(X] A) at the 
desired NE which is derived from the desired PoA. p — 
1 — Prjthe packet dropping probability at desired NE}. Line 
|2]is the calculation of desired NE. The choice of A is based 
on the desired value of PoA, i.e., given e > 0, we could 
accordingly derive the value of a desired sum rate such that 
1 < PoA < 1 + e. Since {Xe,p) is a solution of P(^X), 
line [5] shows how to therefore get the expression of A and D. 
Then at line ID we could solve the equation (l22T i given all the 
values above and get the value of r2- Based on the result of 
r2, the value of ri is calculated. 

Algorithm 1 Parameter Calculation for Incentive Packet Drop- 
ping Scheme 

Input: PoA bound parameter e 

Output: ri and r2 of our proposed incentive packet dropping 
policy in (|2T]l such that 1 < PoA < 1 + e. 

1: Pick any p such that < p <1. 

2: Calculate a desired sum rate A, of which 



~ _ Ae (to - l)Ae 
A - I -, j. 

mp mp 



(25) 



is the desired NE such that 1 < PoA < 1 + e. Note that 

Ae = pX. 

3: Suppose P(J2 Ai) ~ AJ2 + D pass through the point 
{X,p). Then we have 



A^ 



P 



pr2 



(26) 



A - r2 A - r2 

4: Insert the above values of the variables into (|22] | and get 

the value of r2- 
5: Insert the value of r2 into (|26] | and get the value of A. 

Then n = ^ + r2 

Note that (|22]) is a quadratic equation for the parameters A 
and D given the values of all the other variables. But with 
Algorithm [T] we could always find a unique solution as stated 
in Theorem Q 

Theorem 7: Algorithm [T] yields a unique linear packet 
dropping scheme, i.e., unique values for ri and r2 for any 
PoA bound. 
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Proof: 



(m-l)Ae 



After inserting the value p, and A ~ {■^, 



(l22T i at line |4l we have the equality 



(m - l)Ae 
mp 



P 



A. 



} into 



(27) 



.(a + l)(Ae-A*) 



It is obvious that the above is a linear equation of the 
variable r2- And thus we could get a unique solution of r2- 
Therefore, there is a always a unique solution of ri and r2 
provided by Algorithm [T| ■ 

Our proposed packet dropping scheme is similar to the 
Random Early Detection (RED) algorithm. It is simple and 
easy to be implemented with low overhead at the server. Fig. 
m shows an example of our linear packet dropping policy with 
II = 6, m = 2 and a = 2. (fTTI) and ( fT2b are used to calculate 
the utility and PoA. Point A represents A*, which is then 
calculated to be 4 (Ai = A2 = 2). We assume p — 0.9, 
PoA bound parameter e = 0.05. Implementing Algorithm [T] 
with Matlab, we pick Ag = 3.9, we then get ri = 4.3012, 
r2 = 4.622, A = -3.1154, D = 14.4000. Point B represents 
the Nash Equilibrium with our proposed packets dropping 
policy. Point C represents the Nash Equilibrium of the original 
game without packet dropping policy. The shaded area shows 
the cases where packet dropping happens. For the comparison, 
the utilities are shown in the figure and we can see that PoA 
is improved from 1.3396 to 1.0455. 




C(£/c= J.87) 
IUb = 4.11) 
6) 



Fig. 4. An example of our incentive packet dropping scheme. 

X. Uniqueness of NE 

If we use the packet dropping scheme in algorithm [T] we 
are guaranteed that the game Gp always has a NE with the 
desired PoA bound. Now our question is whether the scheme 
yields a unique NE. This is important not only for finding 
out whether our proposed scheme is efficient but also for the 
convergence issues. As surveyed in 121], there are not many 
general results on equilibrium uniqueness. We were unable 
to find any existing theorem that we could use directly to 



prove the uniqueness of NE in our M/M/1 queueing game. 
This makes the analysis of this incentive design problem more 
challenging. 

Theorem 8: There is a unique NE for the M/M/1 Game 
with the linear packet dropping scheme described in Algo- 
rithm [T] 

To prove [8] we first prove the following three lemnias. 

Lemma 1: \A\^ increases monotonically as (A* — Ae) de- 
creases where Ag is the total rate of all users at desired NE 
(as in Algorithm [TJ. 

Proof: Note that (l27T i is equivalent to: 



p2[(a + l)(A*-Ae) + 

This means 

(m - l)Ae 



(m-l)Ae 
mp 



A^(a + l)(Ae-A*) 
mp 



^-(a + 1) I lAl^-p" 

mp{\*-K) \ 



(28) 



Note that as (A* — Ae) decreases, Ag increases and ^ 



increases, so the left-hand side of ( I28I I increase. This implies 
1^1 increases, and thus Lemma [T] ■ 
Lemma 2: VA_,; < ri, ^^'(^j^'-^-') > at Ai = (ri — 

A-.)+- 

Proof: 

Note that P = 1 at ri, then 

^U^{X^,\~^) 

= afi — (a + l)AXiX^i — (a + l)AXf — aX-i 

- (a + l)Xi + aAXifi 
= a/i — aX-i — {a + 1)A,; 

- AA4(a + 1)(A_, + A,) - a/i] 
= an^{a + 1)(A_, + A,) + A_, 

- AX,[{a + 1){X^, + X,) - an] 

= [a/i - (a + 1)(A_, + A,)](l + AX,) + A_, 
= [a/i - (a + l)ri](l + AAj) + A_, 
= [{a + l)X* -{a + l)ri](l + AX,) + A_, 
- (a + 1)(A* - ri)(l + A{r, - A_,)) + A-,; 
Denote (|29] l as g{X-i). Then, 



dx. 



= -{a + l){X* -ri)A+l. 



(29) 



When \A\ is large enough such that ri > X* and \A\ > 

(a + l)(ri-A')' d\^i ^ _ 

From Lemma [T] we know that as Ae gets closer to A*, |^| 
increases. This means that when we design a dropping policy 
with PoA approaching to 1, \A\ could be large enough such 
that n > A* and \A\ > ^^^^^^^^_^,y 

So g{Xi) achieves the minimum value when A_i = ri. Then 

5C/,(A„A_ 



dX, 



> (a + 1){X* ~ ri) + n 

= (a + l)A*-ari ^^^^ 
= Qf/z — ari > 0. 
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Lemma |2] implies that given A_i, Ui{Xi, X-i) which is a 
function of Ai, has a maxima at ri — A_i < Ai < r2 — A_i as 
shown in Fig. |5] 



4 - Local Maximum Point 



1.5 2 2.5 3 3.5 4 



Fig. 5. Illustration of local maximum point of Ui{Xi, A_i) 
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Fig. 6. Illustration of local maximum point of U} and Uf^^^ 



Given A_i, if A_i 



Q + 1 



< ri, then Ui{Xi,X^i) will 



Q + 1 



as shown in 



reach a local maximal point when Xi 

Fig. |6] Note that P{J2^i) = 1 this point. We denote this 
local maximal value as Ul^^^^. Then Ui{Xi, X^i) will reach 
another local maximal point with Xf = {fj.->^-i)a ^ Note that 



P{J2 ^i) < 1- We denote this local maximal value as U^^ax 
Lemma 3: Given A_i, if A_i + < '"i' ^i, 

i,max 

Proof: 
Note that 



< 



where A, 



ui„,,, = ixint,~{xi)~x-.), 

Q + 1 

2 ={pxfn^,-{pxi)~px-.). 



Denote A^ = P^^- Note that A^ ranges from ri to and we 
have ri > A|. Also note that 

m^ax(An"(/^ - - PX-,) > m^ax(AD"(M " A? " A-.) 
since PA_, < A_,. Thus, [/? > C/i„,,„^. ■ 



We show in Lemma|3]that given A-;, Ui{Xi, A_i) achieves 
the maximal point when P < 1 under the condition that A_i + 
^ ^1- When A_i + '•^ ^'^i'"'" > ^'i. there is only one 
maximal point for Ui{Xi, A_i). When A_i + ''^ q+i''" = ^i' 
^Imax ™d Ufmg^x will Overlap, and since ri > A*, this case 
could not result in a NE and thus we do not consider this case. 

Lemma [T] Lemma |2] and Lemma [3] shows that given A_i, 
J7j(Ai,A_i) achieves the maximal point when P < 1. Then 
we will prove the uniqueness of NE based on the expression 
P{Y,X,)=A{\ + X.,)+D. 
Proof of Theorem |S} 

Suppose the Gp has more than one NE. Note that the 
game is symmetric, so there must exist one NE A = 
{Ai, A2, . . . , A„i}, such that 3 i,j,Xi ^ Xj. This means 
there exists a NE A = {Ai,A_,;} and a constant c such that 
A = {Ai + c, A_i — c} is also a NE. 

We insert A — {A^, A-i} into (l24l i and we get: 

[^(A. + A_,) + Dflia + 1)(A* - XU - A?) + AIJ 
= ^A?(a + l)(Al, + A^-A*) 

, (a + l)(A*-Ai,-A,-) + Al, 



A" 



1 



(31) 



[^(A, + A_0 + D] 
We also insert A = {A^ + c, A 



^A{a + l){XU + Xt^X*) 



c} into (|24] | and denote 
. We have: 



" A{\i+c+\-i-c) + D ^ A{\i+\-i)+D 

[A{X, + ~c+X^,-d) + D]^ 

[{a + 1)(A* - XU -c-Xr + d) + X% - 5] 
= A{Xt + ~c){a + 1){X% -c+Xt + ^-X*) 

^ {a + l){X* -XU-K) + ^U-~c 



Af + a 



1 



(32) 



:A{a + l){X% + X1-X*) 



[A[X, + X-i) + Df' 

Note that the right-hand side of dSTT l and ( l32b are the same 
and thus the left-hand side of dsTI) and ( |32| | are equal to each 
other So, 

(a + l)(A*-Al,-A^) + Al, 



K 

^ {a + l){X* ~XU-Xt) + XU-c 
Xl + c 

{Xl + c){a + 1)(A* - Al, - A^) + (A^ + S)A1, 
- XUa + 1)(A* - Al, - A?) + Af (Ai, - d) 

^ dia + 1)(A* - XU - Xt) + ciXU + Xt)^0 
^d[{a + l)X* -a{XU + X'i)] =0 
^ d[a^, - a{XU + Xm - 
^ da[^, - [XU + Xt)] - 
^ ca[^l^ {XU + Xt)] 



A{X^ + X-^)+D 



= 0. 
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Note that - {X'Li + Xf) > 0. This impHes that c = and 
therefore Theorem [8] holds. ■ 

XI. Best Response Dynamics and Convergence 

In this section, we show that the best response dynamic |7|, 
a simple learning mechanism, will lead the queuing game to 
converge to the pure Nash equilibrium. 

Best response dynamic is a straightforward updating rule 
which proceeds as follows: whenever player i has an oppor- 
tunity to revise her strategy, she will choose the best response 
to the actions of all the other players in the previous round. 
Mathematically, for a game G = {N, {Ai}, {Ui}), let a* 
denotes the action of player i in iteration t, 

a\ = arg max Ui{a[, a*Z^). (33) 

a'-eAi 

In general, the best response dynamic is not guaranteed 
to converge. However, if the process does converge, it is 
guaranteed to converge to a NE. Now, we want to investigate 
the convergence of our proposed M/M/1 Game with the packet 
dropping scheme, denoted as Gp in the previous section. 

Theorem 9: Best response dynamic will converge to the 
unique NE for the M/M/1 Game with the proposed packet 
dropping scheme. 
Proof: 

There is an important result about the convergence for 
ordinal potential game as shown in Theorem 21 in fTl: if G is 
an ordinal potential game with a compact action space and a 
continuous potential function, then the best response dynamic 
will (almost surely) either converge to a NE or every limit 
point of the sequence will be a NE. 

We have showed in Theorem [3] that Gp is an ordinal 
potential game, and although the original definition of the 
game does not have a compact action space, the equivalent 
modification as shown in ( fTSl l has a compact action space. 
We can also see that the potential function is continuous. We 
have also proved in Theorem |8] that there is a unique NE. Thus 
Theorem |9] holds . ■ 



^ ^ / / / t t \ 

/'/'///;ti\\\\w 
l^r ,/ / / / y t 1 \ \ \ wxx 



Fig. 7. Quiver plot for a two user example with /i = 10, a = 2, ri = 7.0321 
and r2 = 7.8222. The vector length are scaled to ^ of the original length. 

Figure |2] is an illustration of Theorem |9] by the quiver plot. 
In Figure |7] on the lower triangle of a grid (i.e., the feasible 



operating domain), we plot vector summation for a two-user 
rate control queuing game with fi — IQ, a — 2, ri ~ 7.0321 
and r2 = 7.8222. At each point, the vectors' projections on 
Ai and A2 represent the best response for the corresponding 
users in next iteration. To make the plot neat, the length of 
each vector is scaled to of the original length. The figure 
shows that at each point, the players in the best response 
dynamic move towards the equilibrium point. The length of 
the best response vectors are proportional to the distance from 
the equilibrium point. At the equilibrium point, the step size 
of the movement in the next iteration tends to zero, which 
implies the convergence of the best response dynamic. 

XII. Impact of Arrival Rate Estimation 

For a real system implementation, the server needs to 
estimate the total arrival rate from users in order to apply 
the incentive packet dropping scheme. The packets arrive 
randomly over time, so there will be a difference between the 
estimated total rate and the average total rate. This inaccuracy 
will cause a loss in the PoA. While applying the packets 
dropping scheme, we note that the closer to 1 the desired 
PoA is, the steeper (on the linear part) the packet dropping 
scheme is, and therefore, the greater the impact of estimation 
inaccuracy will be. So as the desired PoA approaches 1, on the 
one hand the PoA of the real system should increase due to 
the implementation of the incentive scheme, but on the other 
hand, the sensitivity to estimation error will reduce the gain 
in PoA. Therefore, the achieved PoA in practice may not be 
arbitrarily close to 1. 

We show this fact in Figure m to [m where we show 
simulation results from a 3-user queue with a = 2. In these 
simulations, we discretize time into slots. The server estimates 
the mean arrival rate from the previous time slot and applies 
the packet dropping function corresponding to a desired PoA 
to all packets in the current slot (the users contribute arrivals 
at a constant rate that corresponds to the equilibrium input for 
this desired PoA). The running time for all the simulations 
is 10^ time slots. Figure [8] and |9] show the simulation results 
of PoA under different service rates for both the sum-utility 
definition and sum-log-utility definition with the instantaneous 
arrival rate to the server as the estimated arrival rate. We can 
see that as the service rate varies from 500 to 5000 packets per 
time slot, the optimal point of PoA (i.e. the lowest achievable 
PoA) is getting closer to 1 because the estimation inaccuracy 
decreases. Also, the empirically achieved PoA is getting closer 
to the desired PoA. 

Note that when the service rate is low, the achievable PoA 
we get could be very bad as shown in figure |8(a)| and |9(a)| In 
these cases, using more history data/longer estimation lengths 
will help to increase the estimation accuracy and improve PoA. 
The comparison results under different estimation lengths for 
/i = 600 packets per time slot are shown in figure [TO] and 
nn These simulation results illustrate a tradeoff between the 
optimal PoA and the overhead in computing and storage: while 
estimating with more history data will increase the estimation 
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accuracy and therefore increase PoA, it increases the overhead 
in terms of computing and storage. 

XIII. Conclusion 

In this paper, we have designed a novel incentive mechanism 
for M/M/1 queueing games with throughput-delay tradeoffs. 
Because the original game yields an inefficient Nash equi- 
librium, we propose to implement a linear packet dropping 
mechanism at the router. We show how the parameters of this 
mechanism can be optimized to ensure system efficiency that 
is arbitrarily close to the social welfare solution. Further, we 
prove that the proposed modification has a unique NE, and that 
the simple best response dynamics converges to this solution. 
Future work could consider extensions of this work to consider 
non-homogeneous users, other queuing models beyond the 
M/M/1 model, more complex arrangements of multiple routers 
in a network, as well as other system issues that may arise in 
practical implementations. 
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Appendix A 
PROOF OF THEOREM □ 

Proof: 



51 log 



a log(]J Xi) + m log(^ - A) 



< 



alog( 



m 
A 



)™+mlog(M-A) 



(34) 



mlog(-)"(M-A)) 



Denote /(A) = X"{fi — A). So maximize (fTTT i is equivalent 
to maximize /(A). Take the derivative of /(A) and let it equals 
0. We get: 

OA a + 1 

Note that equality holds in ( l34l i only when Ai = A2 = • • ■ = 
A„j. This implies A* = ^ = ri,. ■ 

I ni m(Q + l) 

Appendix B 
PROOF OF THEOREM E] 

Proof: (^) 

Suppose = '^*- i^t — mr 

optimal point A** = '^'^t/'^''"' ■ 
Note that 



dU{\,,\'_i) 



0. We get the 



K - A* - Aj - ^ - E. 



Also, VA, < A' 



Qi+l 

dU{\i,X'-i) 



cti + l 



A* 



> 0, which means U{Xi,X'__^) 



so U{\'^,X'_i) > U{X^,X'_,),yX^ e [0,X'i)- Also note that 
C/(A„A'_J = 0,VA, e (A*,M - Ej^»A;). Hence A^ e 
Bi{X'_,). 

Therefore, A' is a N.E. 

(=^) 

Suppose A' is a N.E., Vi, A^ G B,{X'_^). 
VA'_j, consider the following two cases: 

^rE,^.A;.<A- 

Denote A^' = A* - Aj - Then < A^', A'_, > is a N.E., 

A^' G i?,(A'_,). {/(A'/,A'_J = UiX[,X'_,) > 0. So A^ < A* - 
E,^. a;. = A" (if not so, C/(A'„ A'_,) = 0). 

Note that [/(Ai,A'_j) increases monotonically with respect 
to < Aj < X'{. Therefore, A' = A". E A- = A*. 

2) E,^.A;. >A*.- 

Under this case, since E A' > A*, we have Bi{X'_^) — 0, Vi. 
Then J2j=£t ^'3 > A* holds for all i. 

V*,A^<M-E,^.A;.<^-A*=^-^ = ^. So 



< 



a + 1 



</x=>m<a + l. 



However, note that 



X* 



a 



E, , n(m — 1) 
Aj- < . , ^m>a + l 



a + 1 



(35) 



(36) 



increases monotonically with respect to < Ai < A* 



Since ( l35T l and ( |36] | contradict each other, there is no N.E. 
A' such that Ej^^t^ A; > A*. 
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(a) At = 500. (b) ^l = 5000. (c) fj. = 50000. 

Fig. 8. Simulation results of PoA under different service rates of a 3-user system with a = 2 (sum-utility definition). 




(a) tM = 500. (b) n = 5000. (c) /i = 50000. 

Fig. 9. Simulation results of PoA under different service rates of a 3-user system with a = 2 (sum-log-utility definition). 





(a) The estimated total rate got based on the instan- (b) The estimated total rate got by averaging the (c) The estimated total rate got by averaging the 
taneous arrival rate. continuous arrival rates in 10 slots. continuous arrival rates in 100 slots. 

Fig. 10. Impact of estimation length on PoA of a 3-user system with ji = 600, a = 2 (sum-utiUty definition). 
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estimated total rate got based on the instan- (b) The estimated total rate got by averaging the (c) The estimated total rate got by averaging the 
arrival rate. continuous arrival rates in 10 slots. continuous arrival rates in 100 slots. 



Fig. 11. Impact of estimation length on PoA of a 3-user system with fj, = 600, a = 2 (sum-log-utility definition). 



